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(54) Colour image processing apparatus and method 



(57) The invention is to obtain data' under desired 
illumination with a limited memory capacity. 

There is provided an image processing method for 
converting data dependent on a first illuminating light 
into data dependent on a second illuminating light, the 
method comprises storing conversion data for plural il- 
luminating lights of different characteristics., generating 



data indicating the proportion of synthesis of said plural 
illuminating lights of different characteristics, corre- 
sponding to the second illuminating lighLand converting 
data dependent on the first illuminating light into data 
dependent on the second illuminating light, based on the 
conversion data for plural illuminating lights of different 
characteristics and the data indicating the proportion of 
synthesis.- 
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Description 

The present invention relates to an image processing apparatus capable of estimating and correcting color data 
which vary according to the illuminating light, and a method and a recording medium therefor. 

Utilizing the spectral reflectance R(X) of a reflective object, the spectral distribution P(X) of the illuminating light 
and the isochromatic functions: 

x(X),y(X) I z(X) A 

the colorimetric values (X, Y, 2; three stimulation values) of the reflective object under certain illuminating light can be 
represented by: 

x=fcf R(X)-p(M-3F(MdX 

Jvie 

Y=Jc[ R(\)' P(k)-y(X)dX B 
z=Jc\ R(\y P{\)- F(\)dk 

J vie 



wherein the constant k is represented by: 
25 fr=100/J viB P(X)'y(X)dX 



and the integration: 



30 




D 



is executed within the visible wavelength range. 

Consequently the coforimetric values of the reflective object vary according to the variation of the illuminating light. 
Also according to the definitions described above, the spectral distribution P(X) of the illuminating light and the spectral 
reflectance distribution R(X) are required in order to obtain the exact colorimetric values in such state.. 

For this reason, the colorimetric values of the reflective object under arbitrary illuminating light have conventionally 
been determined from the spectral reflectance R(X) of the reflective object and the spectral distribution P(X) of the 
illuminating light. 

The above-mentioned method can be easily executed and can determine the exact colorimetric values in case the 
reflective object. consists of areas of several colors (spectral reflectance R(X)). On the other hand, in case the reflective 
object is for example an image, the object in general has color information in each of finely divided many areas (pixels). 
Consequently there is required a large memory capacity for storing spectral reflectance F\{X) for each pixel, and the 
information ordinarily held for each pixel is the chromaticity values. (X, Y, Z) under a specified condition (specifying 
illuminating light or colorimetric method) or equivalent RGB chromaticity values. In order to determine the colorimetric 
values of the reflective object for the arbitrary illuminating light in the above-described method, there is required the 
spectral reflectance R(X) for each pixel, so that the spectral reflectance R{X) is determined again from the information 
corresponding to the aforementioned colorimetric values. (X, Y, Z) for each pixel or the spectral reflectance R(X) is 
measured again for each pixel. 

In case the information obtained for each pixel is the values corresponding to the colorimetric values (such as the 
aforementioned XYZ values of RGB values) under a specified condition, the colorimetric values of the reflective object 
under arbitrary illuminating light can be determined, . in addition to the above-described method according to the fore- 
going definitions based on the spectral reflectance R(X) of the object and the spectral distribution P{X) of the illuminating 
light, by a method of directly converting the information for each pixel, corresponding to the colorimetric values under 
the specified condition, into the colorimetric values under the arbitrary illuminating light utilizing a matrix, a three- 
dimensional look-up table or a neural network. The conversion function (above-mentioned matrix, three-dimensional 
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look-up table or neural network) is determined tor each of the required plural illuminating lights. 

As explained in the foregoing, the colorimetric values of the reflective object vary according the change in the 
illuminating light. In an image reproducing process or the like, there are often required the colorimetric values of the 
reflective object under arbitrary illuminating light.. 

In case the information obtained for each pixel is the values corresponding to the colorimetric values (such as the 
aforementioned XYZ values of RGB values) under a specified condition, the colorimetric values of the reflective object 
under arbitrary illuminating light can be determined by a method of directly converting the information for each pixel, 
corresponding to the colorimetric values under the specified condition, into the colorimetric values under the arbitrary 
illuminating light utilizing a matrix, a three-dimensional look-up table or a neural network. The conversion function 
(above-mentioned matrix, three-dimensional look-up table or neural network) is determined for each of the required 
plural illuminating lights. 

The above-described method is acceptable in case the number of the required illuminating lights is limited. How- 
ever, for example in the ordinary office environment, the condition of lighting changes in various manner according to 
the kind of the illuminating light source, the time-dependent change thereof, and the change in the state of the incoming 
external light such as the solar light, and it is difficult to prepare or store in advance the conversion functions required 
corresponding to such changes. 

As explained in the foregoing, in order to determine the colorimetric values under arbitrary illuminating light for an 
image or the like that requires information for each of a.large number of pixels, the conventional methods have been 
associated with drawbacks of requiring a large amount of information such as preparing the spectral reflectance for 
each pixel or preparing a large number of conversion functions corresponding to various illuminating light conditions. 

On the other hand, owing to the recent commercialization of various color image processing equipment, the color 
images can be handily processed not only in the special fields such as designing based on the computer graphics but 
also in the ordinary offices. It has however been difficult to consider the color of the printout on the monitor, because 
the color of the image prepared. on the monitor does not in general match that of the printout obtained from the printer 
In order to solve such drawback there has been considered and proposed the color management system. 

The color management system is to cancel the difference in color between the different devices by employing a 
common color space. This system is based oh a principle that a color described by same coordinate values in a same 
color space should always look same and is to match the apparent color by representing alt the colors in a common 
color space and matching the coordinate values. One of the currently employed methods for canceling the difference 
between the devices employs the.CIE-XYZ color space and utilizes the XYZ stimulation values which are the internal 
description coordinates therein. 

However such method may still be insufficient in case the media used for reproduction are different, for example 
the image on a monitor and the printed image. Fig. 27 shows the environment for observing an image on a monitor 
and a printout. In the following it is assumed that an image 1202 same as that on a print 1201 is displayed on a monitor ■ 
1203. 

The printed image or the image displayed on the monitor is not observed under constant ambient light., but the . 
ambient light 1 204 shown in Fig. 27 changes by the opening or closing of a window or the replacement of the illuminating 
light source ; and the image appears differently by such change. Consequently, even when isochromaticity can be 
obtained under certain ambient light, it cannot be preserved even under the same conditions in case the ambient 
lighting condition is altered. 

Though the foregoing consideration has been explained in the comparison of the printed image and the image 
displayed on the monitor it is generally applicable to the comparison of a color presentation by reflecting illuminating 
light and a light-emitting color presentation. More specifically, such phenomenon occurs also in case of taking an object 
such as a person or a sculpture and displaying it on a monitor or presenting as a transmissive display. 

As explained in the foregoing, the appearance of the image varies according the change in the image observing 
environment. In the conventional art employing different media, the images providing isochromaticity under a certain 
situation no longer provide isochromatic feeling by the change in the image observing environment. 

One aspect of the present invention obtains data under desired illumination in a simple manner with a limited 
memory capacity. 

According to another aspect the present invention provides an image processing method for converting data de- 
pendent on a first illuminating light into data dependent on a second illuminating light, the method comprising steps of 
storing conversion data for plural illuminating lights of different characteristics, generating data corresponding to the 
second illuminating light and indicating the proportion of synthesis of the plural illuminating lights of the different char- 
acteristics, and converting the data dependent on the first illuminating light into those dependent on the second illumi- 
nating light, based on the conversion data of the plural illuminating lights of different characteristics and on the data 
indicating the proportion of synthesis. 

Another aspect of the present invention compensates the appearance of the color of the displayed image regardless 

of the observing environment. 
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the amLenS * ^ inV6nti ° n Pr ° VideS * ^ ^ ° f the C ° rrection for 

According to another aspect the present invention provides an image processing method comprising steps of: 

setting a coefficient indicating the ambient illuminating light characteristics according a manual instruction- 
entering image data dependent on an input device; and 

o!h C ° rreC + ti ° n f ° r the ambient N 9 ht ° n the entered image data based on the input device, a display device 



which 



Embodiments of the present invention will now be described with reference to the accompanying drawings, 



Fig. 1 is a chart showing spectral characteristics of light sources of different spectral distributions- 

^JrZli nT Sh 7 mg f Xam . ples ° f co,orime tric values (a*, b*) of a same reflective object illuminated by the light 
sources of different spectral distribution characteristics: 9 

present i^ntion^ ^ Sh ° Win9 6XamP ' eS ° f ' ^ ^ ° bta " ied b * ^ alg ° ri,hm ° f one emb °diment of the 
Fig. 6 is a view showing an example of the colorimetric value estimating system- 

acfeLtl^r' 3 ,eSt C ° nSiSting ° f 77 COlor patches ' em P'°y ed <° r determining the illuminating light char- 
Figs 8, 9, 10 and 11 are views showing examples of the colorimetric value estimating system- 
Fig. 12 is a chart showing the spectral sensitivity of a sensor in the illuminating light sensing unit- 
Fig. 1 3 is a view showing an example of the colorimetric value estimating system- 
Fig. 14 ,s a view showing an example of the user interface for designating the illuminating light 
Fig. 15 is a block diagram showing a color image display system of an embodiment 4- 
Rg. 16 ,s a view showing the flow of data and results of calculations in the calculation of a matrix MTX „■ 
Hg. 17 is a view showing a user interface image frame; 9 
Fig. 18 is a flow chart showing the operations in an ordinary operation mode: 
Fig. 1 9 is a flow chart showing the operations in a user interface mode- 
Fig. 20 is a flow chart showing the operations in the execution of the user interface mode- 
Fig. 21 is a block diagram showing a color image display system of an embodiment 5- 
Fig. 22 is a view showing the user interface image frame of the embodiment 5- 

Figs. 23 and 24 are flow charts showing the operation in the user interface mode of the embodiment 5 

Fla 26 1 1 Sh ° W ? 9 ° pera,ions in the execution ° f the «aer interface mode in an embodiment 6 

Fig. 26 ,s a block diagram showing a color image display system of a variation of the embodiment 1 and 
Hg. 27 is a view showing an image observing environment. 

[Embodiment 1] 

At first there will be explained the principle of estimating the colorimetric values of the reflective object to be 
employed ,n the following embodiments. For converting three stimulation values under certain ilium inat^TlShTinto 
those under another illuminating light, there is known, for example, a conversion method corresponding* the Sanqe 

el tin T\TZ°T e ' "J" 6 C ° IOr adaPtati ° n CQnVerSi ° n (SUCh 38 the me,hod ° f v °" *£.). HcCT'aTSn 
mav bl f^if T , na,Ura ' dayli9ht and mder the type fluorescent lamp, certain colors 

may be a differently ■even .f the color temperature is same. Also certain colors may be felt different even if the iso 
chromaticity is attained by the color adaptation con vers ion. for the colors close to the achromatic color Such^Dhenomena 
are assumed to result from the spectral distribution characteristics of the illuminating light, and the evaluating method 

rfaVsVnS FiaTh eXamp ' e , the ™f ° d ° f eVa ' Uating thS C0 '° r rend6ring ° f the '»** source according^ 
in thf'cni, ( i h IZ I eXamP ' eS ° f ,he li9ht S0UrC6S subsla ntially same in the color temperature but different 

in the spectral distribution. In Fig. 1 , a curve .11 indicates the spectral distribution of a white, ordinary fluorescent 1™ 
while a curve 12 indicates that of a white, high color rendering fluorescent lamp. Fig. 2 shows the wterSSTvaZ 
of a same reflective object, illuminated by the two light sources shown in Fig. 1. In Fig. 2, square marte ^ indicate 
the colorimetric values corresponding to the illuminating light of the spectral characteristics 12 in Fig 1 and the en6 s 
22of line segments corresponding to that of the characteristics 11 in Fig 1 S 
The method for estimating the colorimetric values of the reflective object to be employed in the following embod- 

=cte SESS = inaC^ *" ~ " ^ ^ *" ~ * ^ ^ " 
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In this method, a lighting characteristic matrix CR. defined by the following equation: 

CR = lH ks .CR hr+ (1-lH ks )-CR h! 
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is utilized as means tor estimating and correcting the difference of color, resulting from the difference in the spectral 
distribution characteristics of the illuminating light. 

In the above-mentioned equation, CR hr is a lighting characteristic matrix corresponding to the natural daylight, the 
incandescent lamp or the illuminating light, obtained from a standard light source defined in the Japan Industrial Stand- 
ards or a light source of satisfactory color rendering such as a high color-rendering fluorescent lamp. 

CR hl is a lighting characteristic matrix corresponding to the illuminating light obtained from a light source of low 
color rendering, as defined in the Japan Industrial Standards, such as an ordinary fluorescent lamp. 

Also lH ks is a lighting characteristic coefficient, assuming a value between 0 and 1. 

In case lH ks = 1 , CR coincides with CR hr . The matrix obtained in this case corresponds to the illuminating light of 
satisfactory color rendering, such as the natural daylight, the incandescent lamp, the JIS defined standard light source- 
of the JIS defined high color rendering fluorescent lamp. 

In case IH^ = O, CR coincides with CR hl . In this case, the matrix corresponds the illuminating light of low color 
rendering, such as the JIS defined ordinary fluorescent lamp. 

In case 0 < lH ks .< 1, the lighting characteristic matrix CR corresponds to the illuminating light obtained by mixing 
CR hr and CR h , with a mixing ratio lH ks . 

In this manner it is rendered possible to generate the lighting characteristic matrix corresponding to the illuminating 
lights of various color rendering properties : based on the lighting characteristic matrixes corresponding to the illumi- 
nating lights of different color rendering properties. 

Figs/3, 4 and 5 show the results of estimation of the colorimetric values under arbitrary illuminating light, utilizing 
those under a certain specified condition. In the illustrated cases, the high color rendering illumination of daylight color 
is used as the specified condition. As the arbitrary illuminating light, the case shown in Fig. 3 employs an ordinary white* 
fluorescent lamp (light sourceof low color rendering), while the case in Fig. 4 employs a high color rendering white 
fluorescent lamp (light source of high color rendering), and the case in Fig. 5- employs a daylight white fluorescent lamp 
ot three-wavelength type (light source of medium color rendering). In these drawings, square marks indicate the color- 
imetric values of pixels under the above-mentioned specified condition, while crosses indicate the actually .measured 
colorimetric values under the arbitrary illuminating light and triangles indicates the estimated colorimetric values. 

Fig. 6 shows an example of the flow of the colorimetric value estimating process utilizing the estimating method 
described above. The estimating process can be realized through execution of different process units shown in Fig. 
6, by a CPU according to a program stored in a memory medium and adapted to execute the different process units. 

An image data conversion unit 61 converts the input image data XYZ into estimated image data X'Y'Z' under an 
arbitrary light source according to the following equation, utilizing the lighting characteristic matrix CR calculated in an 
illuminating light characteristic matrix calculation unit 62, wherein the input pixel data XYZ are .normalized to the values 
under a high color rendering light source of a color temperature of 6500°: 
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The lighting characteristic matrix CR is calculated, in the lighting characteristic matrix calculation unit 62 : according 
to the following equation: 



CR=lH k£ -CR hr+ (1-lH ks ).CR h 
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based on the lighting characteristic matrix CR hr stored in a standard lighting characteristic matrix storing unit 63 and. 
corresponding to the illuminating light obtained from the light source of high color rendering such as the natural daylight, 
incandescent lamp, JlS-defined standard light source or JlS-defined high color rendering fluorescent lamp, the lighting 
characteristic matrix CR h! corresponding to the illuminating light obtained from the light source low color rendering such 
as the JlS-defined ordinary fluorescent lamp and a lighting characteristic coefficient IH ks given by a lighting character- 
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istic coefficient instructing unit 64. 

The lighting characteristic matrixes CR hr , CR hl can be obtained by determining the three stimulation values under 
the above-mentioned illuminating lights and those under the standard light source, for example with a test chart con- 
taining 77 color patches as shown in Fig. 7 and executing optimization for example by the attenuated minimum square 
method. 

The lighting characteristic coefficient IH te indicates information on the mixing ratio of the high color-rendering light 
source and the low color-rendering light source as explained in the foregoing, and is set by the lighting characteristic 
coefficient instructing unit 64 according to the kind of the designated illuminating light. 

A plurality of the lighting characteristic coefficients IH ks are prestored in the lighting characteristic coefficient in- 
structing unit 64 corresponding to the kinds of the illuminating lights. 

Fig. 14 shows an example of the user interface; (Ul) relating to the setting of the lighting characteristic coefficient 
The setting of the present embodiment includes, as shown in the user interface 1 40, a selection mode 1 41 and a settina 
mode 142. 

The selection mode 141 displays the kinds of the illuminating lights stored in the lighting characteristic coefficient 
instructing unit 64 in the form of a pull-down menu, for selection by the user. 

The setting mode 142 sets the value of the lighting .characteristic coefficient according to the instruction of the 
user. According to the position of a black triangle mark on a bar, selected by the manual instruction of the user the 
lighting characteristic coefficient instructing unit 64 calculates an sets the value of the lighting characteristic coefficient 
The black triangle mark positioned at the left-hand end of the bar sets 0 as IH ks and that at the right-hand end sets 1 
Also for setting the lighting characteristic coefficient while confirming the effect of the actually set lighting characteristic 
coefficient, there are provided a mode 143 for displaying the color patches in a user interface 145 and a mode 1 44 for 
displaying the original irnage. 

In case the patch 1 43 is selected by the user in the Ul 1 40, the patch data representing the predetermined colors 
are processed according to the lighting characteristic coefficient set on the bar of the Ul 140 and the obtained X'Y'Z' 
data are corrected according to the monitor profile and displayed on the monitor, whereby the user can confirm the 
effect of the set lighting characteristic coefficient. 

Also in case the preview mode 144 is selected by the user in the Ul 140, the original image read according to the 
instruction of the user is processed according to the lighting characteristic coefficient set on the bar and displayed as 
shown in the Ul 146, whereby the user can confirm the effect. 

The above-described embodiment allows to easily estimate and correct the colorimetric values under arbitrary 
light source with a limited memory capacity. 

[Embodiment 2] 

The foregoing embodiment 1 does not execute the correction of the color temperature. The present embodiment 
provides the process including the correction of the color temperature in the process of the embodiment 1 . 

As shown in Fig. 8 : a color temperature conversion unit 81 is added to the colorimetric value estimation process 
shown in Fig. 6, whereby the conversion of the color temperature can be easily achieved. The color temperature 
conversion unit 81 converts the estimated output image data X'Y'Z' into image data X"Y n Z" under the illuminating light 
of a desired color temperature according to the following equation: 





'x"- 




~x>- 


45 


Y" 


= CR 


y' 




Z 1 ' 




z' 



H 



for example by the method of Von. Kries utilizing a color temperature conversion matrix CT corresponding to the in- 
so formation (for example XYZ stimulation values) relating to the arbitrary color temperature and given from the liqhtinq 
characteristic coefficient instructing unit 64. 

Instead of the method of Von. Kries, there may be applied another method of color temperature conversion The 
method of preparation of the color temperature conversion matrix is detailedly explained for example in "Color Engi- 
neering: published by Publishing Office of Tokyo Denki University; Chapter 6 Development of CIE color representation 
55 system". 

In the process shown in Fig. 8, the image data conversion unit 61 and the color temperature conversion unit 81 
are separately provided, but the process may be collectively executed in the image data conversion unit 61 as shown 
in Fig. 9. 
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The colorimetric values under desired illumination can be easily estimated and corrected, including the conversion 
of color temperature, by synthesizing the color temperature conversion matrix CT with the lighting characteristic matrix 
CR. calculated in thelighting characteristic matrix calculation unit 62, as indicated by the following formula to obtain a 
lighting characteristic matrix CR including the color temperature conversion: 



CR' = CT-CR ' 

and converting the image data by such matrix CR'. 
io It is also possible, as shown in Fig. 10, to calculate the color temperature conversion matrix CT in the lighting 

characteristic matrix calculation unit 62 and to execute the matrix calculation based on the obtained result. 

It is also possible to automatically set the lighting characteristic coefficient or the color temperature information 

without the instruction of the user through the user interface as shown in Fig. 14, by providing, as shown in Fig. 11 , a 

light sensing unit 111 and a lighting characteristic coefficient calculation unit 112. 
15 . The light sensing unit 111 is provided with a sensor of sensitivity characteristics for example as shown in Fig. 12 

for measuring the illuminating light, and the BGR output signals of the sensor are processed to obtain individually the 

above-mentioned set values (color temperature, lighting characteristic coefficient). 



20 ■ 



[Embodiment 3] 



Though in the foregoing embodiments 1 and 2, the lighting characteristic matrix is employed for estimating and 
correcting the difference in color resulting from the difference in the spectral distribution of the illuminating lights, the 
present invention is not limited to a method utilizing such matrix and can also be attained for example with a three- 
dimensional look-up table. 

25 ■ in this case, for estimating and correcting the difference in color resulting from the difference in the spectral dis- 

tribution of the illuminating lights, there is employed a lighting characteristic three-dimensional look-up table CR3LUT 
which is defined by the following equation: ~ 

CR3LUT=lH ks -CR3LUT hr +(1-IH ks )-CR3LUT h) J 

wherein CR3LUT hr is a lighting characteristic three-dimensional look-up table corresponding to the natural daylight, 
the incandescent lamp or the illuminating light, obtained from a standard light source defined in the Japan Industrial 
Standards or a light source of satisfactory color rendering such as a high color-rendering fluorescent lamp. 
25 CR3LUT hl is a lighting characteristic three-dimensional look-up table corresponding to the illuminating light ob- 

tained from a light source of low color rendering, as defined in the Japan Industrial Standards, such as an ordinary 
fluorescent lamp. 

Also IH ks is a lighting characteristic coefficient equivalent to that in the foregoing embodiments. 
Fig. 13 shows an example of the colorimetric value estimating system utilizing such colorimetric value estimating 
40 method, wherein an image data conversion unit 131 converts the input image data XYZ into estimated image data 
X'Y'Z 1 under a designated light source according to the following equation: 

(X\ Y' : Z') = CR3LUT (X ; Y : Z) K 

45 

utilizing the lighting characteristic three-dimensional look-up table CR3LUT calculated in the lighting characteristic 
three-dimensional look-up table calculation unit 132. 

It is also possible to estimate and correct the difference in color resulting from the difference in the spectral distri- 
bution of the illuminating lights, employing a function as more general means, and, in such case, the arbitrary illumi- 
50 nating light characteristics are defined by a function (for example CRF). The function CRF satisfies the following rela- 
tionship: 

CRF=CRF(CRF hr ,CRF hl ,IH ks ) L 

55 

which may assume a specific form as in the foregoing embodiments: 
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CRF=IH ks .CRF hr+ (1-!H ks ).CRF h 



M 



or 



CRF=IH te 2 <3RF hr +(1-IH te ) 2 -CRF h 



or 



CRF.IH ks 2 .CRF hr+ (1-IH ks 2 ).CRF h 



O 



wherein CRF hr is a lighting characteristic function corresponding to the natural daylight, the incandescent lamp or the 
illuminating light, obtained from a standard light source defined in JIS or a light source of satisfactory color rendering 
such as a high color-rendering fluorescent lamp; and 

CRF h , is a lighting characteristic function corresponding to the illuminating light obtained from a light source of 
low color rendering, as defined in JIS ; such as an ordinary fluorescent lamp. 

. Based on the above-described function, the colorimetric values XYZ are converted into the values X'Y'Z' as indi- 
cated by the following equation: 

(X\Y\ Z') = CRF(X, Y, Z) p 

wherein IH ks is the lighting characteristic coefficient equivalent to. that in the foregoing embodiments. [Embodiment 4] 
Fig. 1 5 shows an example of the color image display system constituting an embodiment 4 of the present invention 
The upper half in Fig. 1 5 principally shows the flow of the image data and the signal processing, while the lower 

half is composed of data storing units and data calculating units for the signal processing. 

An image data input unit 1101 is connected with a scanner a digital camera or an image memory device and 

receives the image of an object, a printed matter etc. in the form of color image signals R in G in B in . 

The image signals R in G in B in entered by the image data input unit 11 01 are subjected, in an input gamma conversion 

unit 1102, to the correction of the gamma characteristics of the input device providing these image signals utilizing a 

look-up table, and are converted into in-system input signals R in 'G ir /B in ' as indicated by the following equations- 





= LUT Rln 










(G in ) 


Q 




= LUI Vn 


< B ln, 





The above-mentioned gamma conversion LUT is stored in an input device characteristic data storing unit 1106 
and is set. in the input gamma conversion unit 1102 by a system control unit 1111 according to the information from a 
user interface unit 1110. 

The in-system input signals R^G^'B^ are subjected to matrix conversion represented by the following equation 
in an image conversion unit 1103,. thereby being converted into in-system output signals Rout'G^'B^,': 



K out 




'*J' 


G ' 

out 






B ' 

out 







R 



A conversion matrix MTX gh .is determined in an image conversion matrix calculating unit 1109 according to the 
instruction from the system control unit 1111 as will be explained later, utilizing the characteristics of the image input 
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device, image display device and observing ambient light obtained trom the user interlace unit 1110, the input device 
characteristic data storing unit 1106, the display device characteristic data storing unit 1107 and the conversion matrix 
calculation data storing unit 110B, and is set in the image conversion unit 1103. Fig. 16 shows the flow of various data 
and the results of calculations in the above-described operations. 
5 The in-system output signals R out 'G oul 'B oul ' are subjected, in an output gamma conversion unit 1104, to a cor- 

rection according to the gamma characteristics of the display device on which these signals are to be displayed, thus 
being converted into output image signals R out G ou( B out . This conversion is achieved by a look-up table conversion 
utilizing the signals R 0Ul 'G 0Ut 'B ou ,' as represented by the following equation: 
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R out = LUT Rout ( R out ' ) 
G out = LUT Cout ( G ouc ' ) 
B out = LUT Bout (B out ' ) 



The above-mentioned gamma conversion LUT is stored in the display device characteristic data storing unit 1107, 
and is set in the output gamma conversion unit 1104 by the system control unit 1111 according to the information from 
20 the user interface unit 1110. 

The image display unit 1105 is composed of a monitor such as a cathode ray tube or a liquid crystal display device 
and displays an image according to the output image signals R OU |G out B oul . 

In the following there will be explained the function of the system control unit 1111. 
The system control unit 1111 controls the functions of the present system. 
25 Also the system control unit 1111 selects, through the user interface shown in Fig. 17, a gamma conversion look- 

up table matching the input device and a gamma conversion look-up table matching the output device respectively 
from the input device characteristic data storing unit 1106 and the display device characteristic data storing unit 1107 : 
and sets these tables respectively in the input gamma conversion unit 1 1 02 and the output gamma conversion unit 1 1 04. 
Also as shown in Fig. 16, the system control unit 1111 sends, through the user interface shown in Fig. 17, following 
30 data to the image conversion matrix calculation unit 1109, referring to the input device characteristic data storing unit 
H06 ; the display device characteristic data storing unit 1107 and the conversion matrix- calculation data storing unit 
1103^ The above-mentioned data contain an ambient lighting color temperature CCT^, an ambient light luminosity 
IH ks , a conversion matrix M in corresponding to the input device (for converting the in-system input signals R^'G^'B^ 
into S the in-system conversion signals XYZ), a conversion matrix M out corresponding to the display device (for converting 
35 the in-system conversion signals XYZ jnto the in-system output signals R ou ,'G out 'B out , ) : three stimulation values X WI) . 
i y wiii z wiii o* t ne displayed white color of the display device, an ambient light characteristic correcting matrix CR hr 
corresponding to the light source of high color rendering, an ambient light characteristic correcting matrix CR h , corre- 
sponding to the light source of low color rendering, an adaptation ratio s, a matrix M h for converting the three stimulation 
values XYZ of the CI E XYZ into visual RGB values : and a calculation coefficient. 
40 The system control unit causes the image conversion matrix calculation unit 11 09 to calculate the image conversion 

matrix MTX gh and sets the obtained image conversion matrix MTX gh in the image conversion unit 1103. 

The system control data storing unit 1112 stores a flow chart relating to the operations of the system control unit 
1111 and a user interface image, according to which the present system functions. 

The conversion matrix calculation data storing unit 1103 stores calculation coefficients and a lighting characteristic 
45 correcting matrix required tor calculating the image conversion matrix.. 

The image conversion matrix calculating unit 1109 calculates the image conversion matrix MTX gh according to the 
instruction of the system control unit 1111 and based orVthe above-mentioned data. 

Fig. 16 shows the flow of various data and of results of calculations in the calculation of the image conversion 
matrix MTX gh . 

so The image conversion matrix MTX gh is obtained as the product of plural matrixes as indicated by the following 

equation: 



wherein M out is a matrix for converting, for the display device, the in-system conversion signals XYZ into the in-system 
output signals R ou ,'G oul , B 0Ut , 1 whereby the in-system conversion signals XYZ not dependent on the characteristics of 
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the display device are converted into the in-system output signals R 0 ut ,G out'Bout' dependent on (matching) the charac- 
teristics of the display device. 

M jn is a matrix for converting, for the input device, the in-system input signals R^'G^B;,/ into the in-system con- 
version signals XYZ, whereby the in-system input signals R^'G^'B^ dependent on the characteristics of the display 
device are converted into the in-system conversion signals XYZ not dependent on the characteristics of the device 
and the signal conversion in the system can be rendered independent of the individual input, output and display devices. 

M h is a matrix for converting the three stimulation values XYZ of the CIEXYZ color representation system employed 
as the in-system conversion signal system in the present invention into responses R h Gh B h (visual RGB values) at the 
level of receptor of the human eyes (see Foundation of Color Engineering: Asakura Shobo; p.21 6), whereby the image 
signals can be matched with the human characteristics and there can be achieved signal processing simulating various 
processes executed by the human at the image observation. 

CR is a matrix for converting the three stimulation values X D65 Y DG5 Z D65 under the illumination with the standard 
light source (D65) into three stimulation values X ks Y ks Z ks under the ambient lighting. For converting three stimulation 
values under certain illuminating light into those under another iNuminating light, there is known, for example, a con- 
version method corresponding to the change in the color temperature, such as the color adaptation conversion (such 
as the method of Von. Kries). However, as often felt in the image observation under the natural daylight and under the 
daylight type fluorescent lamp, certain colors may be felt differently even if the color temperature is same. Also certain 
colors may be felt different even if the isochromaticity is attained by the color adaptation conversion for the colors to 
the achromatic color. Such phenomena are assumed to result from the spectral distribution characteristics of the illu- 
minating light, and the evaluating method therefor is already known, for example the method of evaluating the color 
rendering of the light source according to JIS-Z-8726 (1990). The matrix CR employed in the present embodiment is 
to correct the difference in the apparent colors resulting from the difference in the spectral distribution of the illuminating 
light mentioned above. In this manner it is rendered possible to correct the difference in the spectral characteristics of 
the ambient illuminating light and to attain better isochromatic feeling. 

The ambient lighting characteristic correcting matrix CR is desirably determined for each ambient illuminating light 
corresponding to the actual ambient situation. For example, this matrix can be obtained by determining the three stim- 
ulation values under the above-mentioned illuminating lights and those under the standard light source, for example 
with a test chart containing 77 color patches as shown in Fig. 7 and executing optimization for example by the attenuated 
.minimum square method. The above-described method can be easily executed in case the number of the required 
illuminating lights is limited to several kinds. In practice, however the condition of lighting changes in various manner 
according to the kind of the illuminating light source, the time-dependent change thereof and the change in the state 
of the incoming external light such as the solar light, and it is difficult, in the above-described method, to determine the 
ambient lighting characteristic correcting matrix corresponding to the ambient illuminating light varying in various man- 
ners. 

On the other hand : in the present embodiment, the ambient lighting characteristic correcting matrix CR is deter- 
mined by the following equation; 

"CR=IH teT CR hr + (1 -IH ks ).CR h( ' u 



in the above-mentioned equation, CR hr is an ambient lighting characteristic correcting matrix corresponding to the 
ambient illuminating light obtained from the natural daylight, an incandescent lamp or a standard light source defined 
in the Japan Industrial Standards or a light source of satisfactory color rendering such as a high color-rendering fluo- 
rescent lamp. CR hr employed in the present embodiment is determined by an isochromatic feeling experiment con- 
ducted with the algorithm of the present invention under respectively light source, utilizing the ambient lighting char- 
acteristic correcting matrixes obtained for the above-mentioned light sources, In the above-mentioned experiment, the 
corrections with the ambient lighting characteristic correcting matrixes corresponding to the above-mentioned light 
sources provided satisfactory results in any of the above-mentioned light sources. Consequently the light sources 
corresponding to the above-described class can be collectively dealt with, and a single correcting matrix can be com- 
50 monly adopted for this class. 

CR h( is an ambient lighting characteristic correcting matrix corresponding to the illuminating light obtained from a 
light source of low color rendering, as defined in the Japanese Industrial Standards, such as an ordinary fluorescent 
lamp. CR hl employed in the present embodiment is determined by an isochromatic feeling experiment conducted with 
the algorithm of the present invention under respectively light source, utilizing the ambient lighting characteristic cor- 
recting matrixes obtained for the above-mentioned light sources. Also in this case, the corrections with the ambient 
lighting characteristic correcting matrixes corresponding to the above-mentioned light sources provided satisfactory 
results in any of the above-mentioned light sources. Consequently the light sources corresponding to the above-de- 
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scribed class can be collectively dealt with, and a single correcting matrix can be commonly adopted for this class. 

Also IH ks is an ambient lighting. characteristic coefficient entered by the user through the user interface unit 1110, 
and assumes a value between 0 and 1 . 

In case IH ks = 0, CR coincides with CR hr The matrix in this case corresponds to the illuminating light of satisfactory 
color rendering, such as the natural daylight, the incandescent lamp, the JIS defined standard light source of the JIS 
defined high color rendering fluorescent lamp. 

In case IH ks = 1, CR coincides with CR hv In this case, the matrix corresponds to the illuminating light of low color 
rendering, such as the JIS defined ordinary fluorescent lamp. 

In case 0 < IH ks < 1, CR becomes a mixture of CR hr and CR h1 with a mixing ratio IH ks , and corresponds to an 
ambient illuminating light obtained by mixing the light source of high color rendering and the light source of low color 
rendering with an arbitrary mixing ratio IH^. The ordinary ambient illuminating light most frequently encountered is in 
this state. 

The above-explained experiment indicate that the light sources of high color rendering and those of low color 
rendering can be respectively grouped, and the ordinary state can be considered as a mixed state of these two groups. 
Assuming such state, an isochromatic feeling experiment was conducted with the above-mentioned algorithm, and 
under various light source, employing the ambient lighting characteristic correcting matrix CR for each light source. 
Also in this case, the correction under such light sources with the ambient lighting characteristic correcting matrix 
corresponding to each of such light source provided satisfactory result. 

In this manner, the above-described algorithm allows to obtain the ambient lighting characteristic correcting matrix 
corresponding to various ambient illuminating lights, utilizing the ambient lighting characteristic correcting coefficient 
IH ks . 

Finally, D is a color adaptation conversion diagonal matrix for effecting color adaptation conversion from a state 
adapted to an observing environment for observing an object, a printed matter etc. to a state adapted for observing 
an image on the display device, and is defined by: 



D = 











0 


0 












0 




0 


V 


0 


0 







wherein (R' hw; G' hwi B' hvy ) are. the visual RGB values of white color under the observing environment/while (R^ w; G hw: 
B hw ) are the visual RGB values of standard while color in the observation of the display device. These visual RGB 
values can be obtained from the three stimulation values XYZ, utilizing the aforementioned matrix M h in the following 
manner: 





= M h 











wherein X' Y' 2' are the three stimulation values of the ambient illuminating iight : which can be obtained through the 
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user interface unit and determined in the following manner, utilizing the ambient lighting color temperature CCT^ and 
the ambient light luminance Y te . At first the colorimetric values (x* w , y' w ) are obtained from the ambient lighting color 
temperature CCT^ according to the following equations: 



/ = 4,6070- 1 0 9 2.9678- 10 6 0.09911- 10 3 

W — CS^7~" + ^r^- 0.244063 

y' w = -3.000- XV+2-870-X' - 0.275 



Then the three stimulation values are obtained according to the following equations, utilizing the above-mentioned 
colorimetric values (x' w , y' w ) and the ambient lighting luminance Y^: 



X 1 -— . Y 

J w 
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On the other hand. X W Y W Z W are the three stimulation values of the standard white color in the observation of the 
display device, and are determined according to the following equations, utilizing the three stimulation values X' W Y' W Z' W 
of the ambient illuminating light and the three stimulation values X wl)l Y wllt Z wllI of the white color on the display device" 
obtained from the display device characteristic data storing unit 107: 

X w = (l-s)*X' w + s-X^ 

w vrm 

Y v = (l-s)*Y' w + s-Y^ z 
Z w = (1-s)* Z' + S »Z_ 



In case of observing an image on the display device, the observer is not considered to adapt to the displayed 
image only but to both the displayed image and the ambient illuminating light in a certain proportion, and the three 
stimulation values X W Y W Z W of the standard white color can be determined from the foregoing equations, wherein s is 
a parameter (adaptation ratio) indicating the proportion of adaptation to the white color on the display, namely the 
influence of the displayed white color, relative to the observing environmental white on the standard white color. The 
adaptation ratio s varies according to the color temperature of the observing ambient light environmental light) and the 
background color of the. image (background color of displayed image). For example, if the background color changes 
in gray scale from black to white, the proportion of adaptation to the ambient light becomes larger as the background 
color becomes closer to black. In the present embodiment the adaptation ratio s assumes a value within a range from 
0.5 to 0.8, corresponding to the above-mentioned conditions. 

In the above-explained manner, the image conversion matrix calculating unit 1 1 09 calculates the image conversion 
50 matrix MTX gh utilizing the aforementioned data and according to the instruction of the system control unit 1111. 
In the following there will be explained the functions of the present embodiment. ' 
In the normal operation mode, the present system functions in the following manner. 
Fig. 18 is a schematic flow chart of the functions of the normal operation mode. 

The image data input unit 1 1 01 acquires the input image signals R in G in B in (s1 01 ), and the input gamma conversion 
unit 1102 executes conversion into the in-system input signals R^G'^B^ as explained in the foregoing, utilizing the 
input device gamma characteristic correcting look-up table which is set in advance corresponding to the current system 
(s102). Then the image conversion unit 1103 converts the in-system input signals R^G^B^ jnto the in-system output 
signals ou & OXi \ utilizing the image conversion matrix which is also set in advance (s103), then the output gamma 
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conversion unit 1104 obtains the output image signals R out G out B out as explained in the loregoing, utilizing the display 
device gamma characteristic correcting look-up table which is also set in advance (s1 04) and the obtained signals are 
displayed by the image display unit 1105 (si 05). 

In case of a change in the input device, the output device or the external environment such as the ambient illumi- 
nating light, the present system can renew, in responsive manner through the user interlace, the input device gamma 
characteristic correcting look-up table, the output device gamma characteristic correcting look-up table and the image 
conversion matrix, which constitute the in-system functions, according to the changed external environment, whereby 
the system can match the apparent colors in the displayed image regardless of the change in the external environment 
of the system. 

Fig. 19 is a flow chart showing the operations relating to the user interface of the present system. 

When the user interface mode is accessed in the present system, a step s202 reads and displays the information 
of the user interface image shown in Fig. 17. On this user interface image, the user enters the input device in an input 
device column, the display device in a display device column, the color temperature CCT ks of the ambient illuminating 
light in an ambient lighting color temperature column, and the ambient lighting characteristic coefficient IH ks in an 
ambient lighting characteristic coefficient column. In this image, a column with a downward arrow is provided in advance 
with plural selection items, which are displayed as a pull-down menu upon selection of the arrow mark. A desired item 
can be entered by selection in the pull-down menu. The above-mentioned information are fetched in a step s203. A 
step s204 discriminates the selection of an execution button, and, if it is selected, the sequence proceeds to a step 
s205 to execute the flow shown in Fig. 18. If the execution button is not selected, the sequence proceeds to a step 
s207 for discriminating the selection of an end button. It it is selected, the present operation mode is terminated. If the 
end button is not selected, the sequence returns to the step s203. A step s206 discriminates the completion of oper- 
ations, and, if completed, the sequence returns to the step s203. If not completed, the sequence waits in a stand-by 
state in the step s206. The operations in the user interface mode of the present system are executed in the above- 
explained manner. • ' 

In the following there will be explained, with reference to Fig. 20, the operations in the execution of the user interface 

mode. 

When the execution o.f the user interface mode is selected, the system control unit 1111 reads, in a step s301. a 
program realizing the flow of the user interface mode from the system control data storing unit 1112. and initiates 
operations according to the program. At first a step s302 specifies the image input device and selects the corresponding 
input device gamma characteristic correcting look-up table from the input device characteristic data storing unit 1106 : 
and sets the look-up table in the input gamma conversion unit 11 02. Also the conversion matrix M in: for converting the 
in-system input signals RY.G^B'jn into the in-system conversion signals XYZ. is selected according to the image input 
device from the input device characteristic data storing unit 1106 and is set in the image conversion matrix calculating 
unit 1109. Then a step s303 specifies the image display device and selects the corresponding display device gamma 
characteristic correcting look-up table from the display device characteristic data storing unit 1107 and sets the look- 
up table in the output gamma conversion unit 1 1 04. Also the conversion matrix M out , corresponding to the image display 
device and adapted for converting the in-system conversion signals XYZ into the in-system output signals RouiG'outB'^,' 
and the three stimulation values X wlll Y wm 2^ /ln of the displayed white color of the display device are selected from the 
display device characteristic data storing unit 1107 and are set in the image conversion matrix calculating unit 1109. 
* Then a step s304 sets the ambient lighting color temperature CCT ks , the ambient lighting luminance Y ks and the ambient 
lighting characteristic coefficient IH ks: obtained through the user interface, in the image conversion matrix calculating 
unit 1109. A step s305 sets the lighting characteristic correcting matrix and various calculation coefficients, required 
for the calculation of the image conversion matrix, from the conversion matrix calculation data storing unit 1108 in the 
image conversion matrix calculating unit 1 1 09. Then a step s306 calculates the image conversion matrix MTX gh by the 
. aforementioned method, utilizing the data set in the image conversion matrix calculating unit 1109 in the foregoing 
steps s302 : s303 : s304 and s305, and sets the obtained matrix MTXg h in the image conversion unit 1103. 

Then the image entered by the image data input unit 1101 is converted in the input gamma conversion unit 1102 : 
the image conversion unit 1103 and the output gamma conversion unit 1104, based on the input device gamma char- 
acteristic correcting look-up table, the display device gamma characteristic correcting look-up table and the image 
conversion matrix set in the steps s302, s303 and s306, and the converted image is displayed on the image display 
unit 1106. These operations are same as those in the foregoing normal operation mode. The above-explained opera- 
tions are executed in steps s307 t s308, s309 : s310 and s311. 

After these operations, the sequence returns to the aforementioned operations relating to the user interface. 
The system of the present embodiment, provided with the means for obtaining information from the user and the 
image conversion matrix calculating means, through the execution of the above-described operations, allows to match 
the apparent colors on the object, the print and on the display regardless of the external environment such as the input 
device, the display device and the ambient lighting. 

■In the foregoing embodiments, the color temperature, luminance and color rendering characteristic of the ambient 
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illuminating light are entered through the user interface unit 1110, but there may be provided an ambient light sensing 
unit 1114 for automatically measuring and setting these parameters of the ambient illuminating light, as shown in Fig 
26. In the configuration shown in Fig. 26, the ambient light sensing unit 1114 employs a sensor of the sensitivity char- 
acteristics shown in Fig. 1 2, and the output signals RGB of this sensor are processed to individually obtain the above- 
mentioned parameters. This configuration is substantially same as the foregoing configuration except that the above- 
mentioned parameters are obtained not from the user interface unit 1110 but from the ambient light sensing unit 1114, 
and the functions are also substantially similar. Also the above-mentioned set values may be entered independently 
from suitable means. Also such configuration allows to set the characteristics of the ambient illuminating light by meas- 
urement. 

Furthermore, the present embodiment is capable of satisfactory signal conversion, taking into consideration the 
spectral characteristics of the observing environment and the color adaptation characteristics of the observer (influence 
of both the displayed white color and the ambient light white color on the standard white color in the observation of the 
displayed image), by executing the image conversion based on the spectral characteristics of the ambient illuminating 
. light and also executing the color adapted conversion based on the standard white color in the observation of the 
displayed image, obtained from the ambient illuminating light and the white color of the display device. 

The foregoing embodiment employs the theory of von Kries in the color adapted conversion, but there may also 
be applied other color adaptation estimating theories. Also the present invention is applicable to various hardware 
configurations and sequence processes corresponding thereto. Such sequence process may be realized into a logic 
circuit or a software or an algorithm within the aforementioned scope of the present invention, and the hardware or the 
apparatus can be realized according to. such algorithm. 

The present invention is further applicable to a copying apparatus or a printer provided with a previewing function, 
for displaying the image to be printed in advance on a monitor. • 

[Embodiment 5] 

In the foregoing embodiment 4. the image conversion matrix to be utilized in the image conversion unit 1103 is 
obtained in the user interlace unit, by the determination in the image conversion matrix calculating unit 1109 for each 
setting of the input device, display device, ambient lighting color temperature, ambient lighting luminance and ambient 
lighting characteristic coefficient. 

■ The input device, the display device and the ambient illumination may be limited to a certain number of combina- 
tions. In such case, such combinations are finite and the image conversion matrix may be stored corresponding to 
each of such combinations. 

The present embodiment is suitable for such case, and is realized by adding an image conversion matrix storing 
unit 1113 to the color image display system of the embodiment 4 shown in Fig. 15. 
Fig. 21 shows the color image display system of the present embodiment. 

The image conversion matrix storing unit 1113 stores the matrix, calculated in the image conversion matrix calcu- 
lating unit 1109, according to the instruction of the user interface unit 1110. According to the instruction from the user 
interface unit 1110, the system control unit 1111 stores the image conversion matrix from the image conversion matrix 
calculating unit 1109 to the image conversion matrix storing unit 1113 or setting the matrix therefrom in the imaqe 
conversion unit 1103. 

The functions of the present embodiment are same as those of the foregoing embodiment 4, except for the addition 
of a step of storing the calculated image.com/ersion matrix and a step of selecting and setting the stored imaqe con- 
version matrix. 

Fig. 23 is a flow chart of the operations relating to the user interface of the present embodiment. 
Also Fig. 22 shows the user interface image of the present embodiment. 

In comparison with the user interface image of the previous embodiment, there is added an input area for registering 
and selecting the external environment of the system. In this image, a column with-a downward arrow is' provided in 
advance with plural selection items, which are displayed as a pull-down menu upon selection of the arrow mark A 
desired item can be entered by selection in the pull-down menu. 

In the following there will be explained the functions of the present embodiment with reference to Fig 23 When 
the user interface mode is accessed in the present system, a step s202 reads and displays the information- ofthe user 
interface image shown in Fig. 22. On this user interface image, the user enters the input device in an input device 
column, the display device in a display device column, the color temperature CCT^ of the ambient illuminating light in 
an ambient lighting color temperature column, the luminance Y ks of the ambient illuminating light in an ambient light 
luminance column and the ambient lighting characteristic coefficient IH ks in an ambient lighting characteristic coefficient 
column. 

Otherwise the user enters, in an input column for the system external environment, the system environment data 
(input device, display device and image conversion matrix mentioned above) stored in advance by means to be ex- 
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plained later. 

The above-mentioned operations are selected according to discrimination in a step s20B whether the system 
external environment is selected. 

If the external environment is not selected, a step s203 enters the input device from the input device column, the 
display device from the display device column, the color temperature CCT ks of the ambient illuminating light from the 
ambient-lighting color temperature column, and the luminance Y ks of the ambient lighting from the ambient light lumi- 
nance column and the ambient lighting characteristic coefficient IH ks from the ambient lighting characteristic coefficient 
column. 

If the external environment is selected, a step s210 fetches the system environment data. 

Then a step s204 discriminates the selection of an execution button, and, if it is selected, the sequence proceeds 
to a step s205 to execute the operations of the flow chart. If the execution button is not selected, the sequence proceeds 
to a step s207 for discriminating the selection of an end button. If it is selected, the present operation mode is terminated. . 
If the end button is not selected., the sequence proceeds to a step s208. A step s206 discriminates the completion of 
operations, and, if completed, the sequence proceeds to a step s209. If not completed, the sequence waits in a stand- . 
by state in the step s206. 

The step s209 discriminates whether the current external environment data of the system are to be stored, and : 
il not, the sequence returns to the step s20B. 

In case the data are to be stored, a name for storing the external environment is read from the user interface image, 
and the external environment data are stored under such name. 

The operations of the present system relating to the user interface mode are executed in the above-explained 
manner. 

In the following there will be explained the operations in the execution of the user interface mode, with' reference 
to Fig. 29, which is a flow chart showing the operations in the execution of the user interface mode in the present 
embodiment. 

The operations are basically same as those of the embodiment 4, except for the presence of a branched step in 
which the image conversion matrix is selected from the image conversion matrix storing unit 1113 and is set in" the 
image conversion unit 1103. 

The system of the present embodiment, through the above explained operations and owing to the presence of the 
selection means by the user, the image conversion matrix calculating means and the storing means, allows to easily 
match the apparent colors of the object, the printed image and the image displayed on the display device under a given 
environment regardless of the external environment of the system such as the input device, display device and ambient 
illuminating light. 

[Embodiment 6] 

As in the embodiment 5 : the present embodiment 6 stores the image conversion matrix corresponding to the 
combination of the input device, display device and ambient lighting. 

Fig. 21 shows the color image display system of the present embodiment, which is obtained by adding an image 
conversion matrix storing unit 1113 to the color image display system of the embodiment 4 shown in Fig. 15. 

The~difference between the present embodiment and the embodiment 5 lies in a fact whether the image conversion 
matrix to be employed is directly designated in the user interface unit by the external environment input column. 

In contrast to the embodiment 5 utilizing direct designation, the system control unit 1111 of the present embodiment- 
is provided with a step of analyzing the input data of the user interface, and, if the result obtained in such analysis is 
same as a mark attached to the previously prepared and stored image conversion matrix, the previous matrix is selected 
without calculation of the matrix anew. 

In case the image conversion matrix is calculated anew in the present embodiment, the result of analysis ol the 
user interface input data is displayed and stored in the image conversion matrix storing unit 1113. 

The user interface image employed in the present embodiment is similar to that of the embodiment 1 shown in 
Fig. 17. 

The operations of the present embodiment are substantially similar to those of the foregoing embodiment 5, except 
tor the above-described operations. 

The system of the present embodiment, through the above-explained operations and owing to the presence of the 
image conversion matrix calculating means and the storing means, and the user interface input data analyzing means, 
allows to easily match the apparent colors of the object, the printed image and the image displayed on the display 
device under a given environment regardless of the external environment of the system such as the input device, 
display device and ambient illuminating light. 

As explained in the foregoing embodiments, in order to match the apparent colors in the display on the monitor 
and in .the print, it is necessary to execute the conversion of the color signals in lull consideration of the characteristics 
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(spectral characteristics, color rendering property etc.) of the ambient iight. The foregoing embodiments are different 
in the method of setting the data required for such conversion, and the intention of the present invention is, in an image 
display method, an image display apparatus and an image display system, to provide means for setting the input 
device, display device and ambient illuminating light, and color signal conversion means capable, based on the char- 
acteristics of thus set input device, display device and ambient illuminating light, of executing a color signal conversion 
based on the characteristics of the input device, a color signal conversion based on the characteristics of the ambient 
illuminating light, a color signal conversion including the color adaptation based on the characteristics of the ambient 
illuminating light and the displayed white color of the display device, and a color signal conversion based on the char- 
acteristics of the display device. 

More specifically, based on the information (chromaticity, color temperature or spectral intensity (luminosity)) on 
the illuminating light (ambient light), there are obtained information (chromaticity, XYZ stimulation values etc.) on the 
white color sensed under such illuminating light and information (for example two-dimensional matrix) for converting 
other colors, and the color signal conversion is executed utilizing these information. 

According to the foregoing embodiments, there is achieved precise conversion of the color signals corresponding 
to various ambient light sources, thereby attaining sufficiently precise is'ochromaticity for the display on the monitor 
and for the printed image. 

Also the present invention is applicable to various hardware configurations and sequence processes corresponding 
thereto. Such sequence process may be realized into a logic circuit or a software or an algorithm within the aforemen- 
tioned scope of the present invention, and the hardware or the apparatus can be realized according to suchaigorithm. 

[Other embodiments] 

The present invention is applicable not only to a system consisting of plural equipment (for example host computer 
interface equipment, reader, printer elc.) but also to an apparatus consisting of a single equipment (for example a 
copying apparatus or a facsimile apparatus). 

The present'invention also includes an embodiment of supplying a computer in an apparatus or a system, so 
connected with various devices as to realize the functions of the foregoing embodiments, with program codes of a 
software for realizing the functions of the foregoing embodiments whereby the effects of the present invention are 
achieved by the functions of such various devices according to the program stored in the computer of such system or 
apparatus. 

In such case, the program codes themselves of the above-mentioned software realize the functions of the foregoing 
embodiments, and the program codes themselves and means for supplying the computer with such program codes, 
for example a memory medium storing such program codes, constitute the present invention. 

The memory medium for storing the program codes can be, for example, a floppy disk t a hard disk., an optical disk, 
a magnetooptical disk, a CD-ROM ; a magnetic tape, a non-volatile memory card or a ROM. • 

Also the present invention naturally includes not only a case in which the functions of the foregoing embodiments 
are realized by the execution of the supplied program codes by the computer but also a case in which the functions 
are realized by the cooperation of the program codes with the operating system of the computer or another application 
software. 

Furthermore the present invention includes a case in which the supplied program codes are stored in an expansion 
board of the computer or a memory provided in an expansion unit connected to the computer and a CPU or the like 
provided in such expansion board or expansion unit executes all or a part of the actual process according to the 
instruction of such program codes thereby realizing the functions of the foregoing embodiments. 

The present invention is not limited to the foregoing embodiments but is subject to any and all modifications and 
variations within the scope of the appended claims. - 



Claims 

1. An image processing method for converting data dependent on a first illuminating light into data dependent on a 
second illuminating light, comprising steps of: ' . 

storing conversion data for plural illuminating lights having different characteristics; 

generating data indicating the proportion of synthesis of said plural illuminating lights having different charac- 
teristics, corresponding to said second illuminating light; and 

converting data dependent on said first illuminating light into data dependent on said second illuminating light, 
based on said conversion data for plural illuminating lights having different characteristics, and said data in- 
dicating the proportion of synthesis. 



16 



70 



EP 0 891 077 A2 

2. An image processing method according to claim 1, wherein said plural illuminating lights are different in color 
rendering property. 

3. An image processing method according to claim 1 , wherein said data indicating the proportions ot plural syntheses 
are stored in advance according to the kinds of the iHuminatirig light. 

4. An image processing method according to claim 3, wherein the kind of said second illuminating light is designated 
by the user and said data indicating the proportion of synthesis are selected according to said designated kind of 
the second illuminating light. 

5. An image processing method according to claim 1, wherein said data indicating the proportion of synthesis are 
generated according to a manual instruction of the user. 

6. An image processing method according to claim 1, wherein said data indicating the proportion of synthesis are 
is generated according to the output from a sensor for measuring the illuminating light. 

7. An image processing method according to claim 1 wherein said conversion data are matrix data. 

B. An image processing apparatus for converting data dependent on a first illuminating light into data dependent on 
20 asecond illuminating light, comprising: 

storage means for storing conversion data for plural illuminating lights having different characteristics; 
generation means for-generating data indicating the proportion of synthesis of said plural illuminating lights 
having different characteristics, corresponding to said second illuminating light;- and 
25 conversion means for converting data dependent on said first illuminating light into data dependent on said 

second illuminating light, based on said conversion data for plural illuminating lights having different charac- 
leristics. and said data indicating the proportion of synthesis. 

9. A computer readable recording medium storing a program said program comprising the steps of: 

storing conversion data for plural illuminating lights having different characteristics: 
generating data indicating the proportion of synthesis of said plural illuminating lights having different charac- 
teristics, corresponding to said second. illuminating light: and 

converting data dependent on said first illuminating light into data dependent on said second illuminating light.. 
35 . based on said conversion data for plural illuminating lights having different characteristics, and said data in- 

dicating the proportion of synthesis. 

10. An image processing method comprising steps of: 

40 setting an ambient lighting characteristic coefficient according lo a manual instruction: 

inputting image data dependent on an input device; and 

effecting correction for the ambient lighting on said inputted image data based on said input device., a display 
device and said ambient lighting characteristic coefficient, thereby achieving conversion into image data de- 
pendent on said display device. 

45 

11. An image processing. method according to claim 10, wherein said correction for the ambient lighting is achieved 
by correction of color rendering based on said ambient lighting characteristic coefficient. 

12. An image processing method according to claim n, -wherein said correction of color rendering is achieved by a 
so weighted process on said conversion data corresponding to the plural light sources having different color rendering 

properties, based on said ambient lighting characteristic coefficient. 

13. An image processing method according to claim 10 : wherein matrix coefficients relating to said correction for the 
ambient lighting are calculated according to said ambient lighting characteristic coefficient. 

55 

14. An image processing method according to claim 13, wherein said calculated matrix coefficients are registered 
according to a manual instruction. 
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15. An image processing method according to claim 10, further comprising a step of: 
■ setting the color temperature and the luminance of the ambient light; 

wherein said correction for the ambient light is achieved by a color adapted conversion according to said color 
temperature and luminance. 

16. An image processing apparatus comprising: 

setting means for setting an ambient lighting characteristic coefficient according to a manual instruction: 
input means for entering image data dependent on an input device; and 

conversion means for effecting correction for the ambient lighting on said entered image data based on said 
input device, a display device and said ambient lighting characteristic coefficient, thereby achieving conversion 
into image data dependent on said display device. 

1 7. A computer readable recording medium storing a program for executing an image processing method, said program 
comprising steps of: * , 

setting an ambient lighting characteristic coefficient according to a manual instruction; 
inputting image data dependent on an input device; and 
20 effecting correction for the ambient lighting on said inputted image data based on an input device, a display. 

device and said ambient lighting characteristic coefficient, thereby achieving conversion into image data de- 
pendent on said display device. 
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